Prior art field-sequential electronic stereoscopic video or television displays have suffered from a number of shortcomings, principally relating to high cost of manufacture and compromises in image quality. Typically, flickerless electronic stereoscopic video displays have been priced at tens of thousands of dollars. Given the approach in which the left and right images share a single channel, the resultant stereoscopic image has noticeably poorer image quality than that of one planar video image which solely inhabits the channel. Other field-sequential electronic stereoscopic video or television systems, which have a low user price, have made a serious performance compromise since they have a flickering display. While it is true that such products are attractively priced, they are not suitable for professional applications, and will have limited acceptance as an entertainment medium despite their novelty because of the obtrusive flicker.
In order for a field-sequential stereoscopic video display to be flickerless, each eye must see approximately the same number of fields in a unit of time as both eyes see in a non-stereoscopic ("planar") display. This requires a doubling of the vertical frequency, or the refresh rate. In the United States, and other NTSC countries, in which the refresh rate is approximately 60 fields/second for a standard planar video system, it is convenient to make a stereoscopic display which refreshes at 120 fields/second, rather than say, 110 or 140 fields/second, or some other arbitrary number. In this case, when using the proper selection device, each eye will see 60 fields/second. In PAL systems, which employ a standard refresh rate of 50 image fields/second, a rate of 100 fields/second is required for a flickerless stereoscopic display.
StereoGraphics Corporation produced the original flickerless electro-stereoscopic product which was first shipped in 1983, covered by U.S. Pat. No. 4,523,226, in which a form of image compression is used so that both left and right views share the existing video bandwidth. In this case the two images are squeezed by a factor of two in the vertical direction, as shown FIG. 8 of the present application. The resultant above-and-below sub-fields, 802 and 803, are located adjacent to each other within a single video field 801, and separated by a sub-field blanking interval, 804. These sub-field images are expanded in the vertical direction upon playback and displayed at twice the field frequency. In some applications, a scan converter, or line doubler, made by JVC or Sony, has been used to double the number of lines displayed to produce a smoother appearing raster. However, this approach exacerbated the stairstepping or jaggies artifact of diagonal lines. The above-and-below sub-field technique, in the first generation of manufacture, involved a modification to cameras to allow them to function at 120 fields/second, as described in Lipton et al's U.S. Pat. No. 4,583,117.
In the second generation, a digital device, manufactured for StereoGraphics Corp. by Fortel, was used to create the vertically compressed above-and-below sub-field format, and to restore this format to a field sequential 120/fields/second display. This allowed off-the-shelf, unmodified cameras to be employed, making the system more versatile and allowing the user to select from amongst a variety of video cameras. However, the analog to digital to analog compression/decompression device was costly to manufacture, and the technique intrinsically produced artifacts, such as the aforementioned stairstepping for diagonal lines and a reduction in vertical resolution. These artifacts were visually unpleasant.
Nevertheless, the second generation product had these virtues: it worked, was recordable with standard equipment, and produced a fair quality stereoscopic image, and it was the only thing of its kind one could purchase. But the systems' image quality was not as good as a comparable planar image, especially when viewed on large screens.
Interestingly, the technique of U.S. Pat. No. 4,523,226 has become the standard approach used by most graphics computers and graphics boards manufacturers who make 120 field/second stereo-ready products. In these applications, the technique works nicely because of the greater bandwidth available in computer systems, with their greater number of lines per field, compared with video displays.
An alternative technique, which is described in U.S. Pat. No. 4,562,463, to Lipton, produces a flickerless effect for a stereoscopic image, but does so by doubling the number of fields at playback. It does not use image compression; rather the camera set-up uses a video switch to switch between the perspective viewpoints at field rate. Upon playback the fields are stored in memory and read back at twice the rate at which they were stored. If played back in the proper sequence, the result is a flickerless field-sequential stereoscopic image. In the case of Lipton et al., U.S. Pat. No. 4,523,226, the multiplexing approach is based on a compromise in terms of spatial sampling in the vertical direction, leading to a reduction in vertical resolution, with the attendant artifact of stairstepping. On the other hand, in Lipton, U.S. Pat. No. 4,562,463, a compromise has been made in sampling in the temporal domain, which may produces image jitter, and possibly spurious temporal parallax, as described in Lipton, "Foundations of the Stereoscopic Cinema (Van Nostrand, 1983).
Other field-sequential, flickerless stereoscopic systems have been demonstrated, usually at trade shows. Philips and others have shown dual bandwidth systems using interlocked laser disc players supplying left and right images to two video projectors. The polarized light method of image selection is used in such a set up.
Ikegami has shown a dual bandwidth system, based on the NTSC protocol, in which signals from two NTSC cameras are combined in a storage device and then read out at twice the rate at which they were stored. The result is a signal with twice the normal bandwidth. This device was shown the conventions of the National Association of Broadcasters and the Society of Motion Picture and Television Engineers in 1990, with a price quoted at $150,000 and an unspecified delivery date. A similar approach was employed by NHK and described in the SMPTE Journal, February, 1990, in an article by Isono and Yasuda, entitled "Flicker-Free Field-Sequential Stereoscopic TV System and Measurement of Human Depth Perception." In the case of the last two systems described above, no attempt was made to produce a multiplexed signal which could be transmitted over existing NTSC transmission lines or which could be used with a single NTSC recording medium such as a video tape or disc player.
Other flickerless field-sequential plano-stereoscopic systems requiring individual selection devices have been demonstrated, but with the exception of the StereoGraphics product, none have been compatible with established video protocols.
In order for a field-sequential, flickerless stereoscopic video system to be successful for industrial, scientific, education, and entertainment uses, it must be compatible with the existing protocol and hardware infrastructure vis-a-vis cameras, transmission, recording, playback, and post-production. For example, it must operate with a single recorder rather with two electronically interlocked machines, as is the case for the Ikegami system mentioned above. Moreover it must not be priced a great deal more than a planar video system used for the same application. For example, if a teleoperations system is used in an underwater remotely operated vehicle, and a typical video system is priced at $15,000, it is unlikely that users will spend $150,000 or even $40,000 for a substitute stereoscopic video system, whatever its performance.
There is a distinction that must be made between upwardly and downwardly compatible stereoscopic video systems which involves multiplexing techniques, discussed in "Compatibility of Stereoscopic Video Systems with Broadcast Television Standards," by Lipton in SPIE, Vol. 1083, 1989. In the systems described in cited U.S. Pat. Nos. 4,523,226 and 4,562,463, no attempt was made to produce a downwardly compatible product. These systems are not downwardly compatible in the sense that their transmitted or recorded signals, when played back on a conventional monitor, will result in images appearing to be compressed or scrambled. An example of a downwardly compatible approach is the NTSC colorplexing protocol in which color information is added to the existing black-and-white transmission allowing black-and-white sets to receive a color signal which can produce an image of unimpaired quality. NTSC color information is transparent to monochrome sets, but to a color set the information produces a color image.
Suggestions for downwardly compatible stereoscopic video systems have been made by several workers, including Yoshimura in U.S. Pat. No. 4,772,944, and Yamada in U.S. Pat. No. 4,743,965. However, a study of these multiplexing techniques reveals that, while possibly downwardly compatible vis-a-vis existing receivers or transmission systems, they would not result in signals which could be recorded or played back on most conventional video tape recorders or laser disc players. The additional information required for the reconstruction of the complete stereoscopic image might be beyond the capability of professional high bandwidth recorders. It would be necessary to produce a new type of video tape recorder to take into account the stereoplexed information. This would impose an enormous burden on consumers because tens of millions of video tape recorders are in their hands. The concept of compatibility, in this the age of the consumer video tape recorder, takes on additional complications that did not exist at the time of the introduction of the downwardly compatible colorplexed video protocols now employed throughout the world. As a matter of fact, the video tape recorders which have been introduced subsequently have had to take into account the additional color information and make special provision for its recording.
Today, the compatibility requirements are more severe than in prior years, and it may well be technically impossible to create a stereoscopic multiplexing technique which is compatible with the entire video infrastructure, which incorporates stereoscopic information which is transparent to existing receivers.
We expect that the upwardly compatible invention of the present disclosure will meet the requirements of the marketplace by producing a multiplexed video signal which can be displayed as a stereoscopic image of high quality, and which can be recorded and played back on existing professional and consumer video recorders and laser disc devices. The multiplexed video signal of the invention may be distributed by existing transmission schemes including satellite broadcast, through-the-air broadcast, and cable distribution.
Despite the fact that the multiplexed signal of the invention appears as a compressed or scrambled image when displayed on a non-stereoscopic ("planar") monitor, it may still be viable as a consumer product. These days there are many more means for delivering a video signal than in the days when color television was introduced. Then only VHF broadcast channels were available. Now there is also UHF, cable, home video tape recorders (VTRs), and the possibility of satellite broadcast. With so many more channels or means of distribution of video programs, there is the distinct possibility that a portion of some of these means might be dedicated to stereoscopic video.
The issue of downward compatibility is of less importance in an industrial, scientific or military application than it is in a consumer application, but it is a matter that needs to be considered.
There are many applications for a stereoscopic video system in industry, science, medicine, education, and the military. These users may be willing to pay more than home users for such capability, but eight years of experience of marketing such products indicate that they are not willing to pay a lot more. These days closed circuit color television systems using solid state cameras and good quality monitors have reached a high level of performance and image quality, at a low price.
Therefore it is desirable to design a field-sequential stereoscopic television system whose signal can be sent over a transmission line with no more bandwidth than that allocated for a planar signal, and can be recorded on an unmodified video tape recorder, so that the recorded signal may be played back on a stereo-ready television set or monitor. The stereoscopic images must be more or less subjectively equivalent in image quality to a normal planar video display. That means that each of the two perspective views must have more or less the same image quality as a non-stereoplexed planar image. This must be accomplished at a reasonable cost of manufacture and sale price.
Various multiplexing techniques have been employed to incorporate additional video information within a single channel. One such product, known as the Comband system, was announced in the 1980's by General Electric, and is still offered by the Comband Company. The Comband system embodies a multiplexing technique of the type described in U.S. Pat. No. 4,533,936, to Tiemann, et al. The system uses a complex scheme in which the luminance signal is filtered to two and a half megahertz, or approximately half that of the normal NTSC bandwidth. An additional channel is added to the signal, and it is also compressed. The product allows a doubling of the number of channels available in a cable system.
Yet another approach is one which is described by G. R. Kennedy in "Weather Satellite Picture Processor, "Wireless World, Vol. 86, 1980, No 1533, in which two channels are incorporated within a single video field. In this case the images are located side-by-side, but there is no compression, and the images maintain their usual aspect ratio. Apparently this approach was used for convenience to save bandwidth and also to allow for fax transmission in certain circumstances.
Compression in the time domain has been used. One example of this is described in U.S. Pat. No. 4,027,333 to Kaiser et al., in which alternate fields are encoded with alternate images. The images are then separated on an alternate field basis, and interpolation is used to synthesis missing field information. This device was developed at C.B.S. Labs and then by Thompson CSF. It was also part of a system discussed by Liston Abbott in an article in the February, 1979 SMPTE Journal, Volume 88, entitled "Transmission of Four Simultaneous Television Programs Via A Single Channel." Multiplexing in the time domain may be the most costly approach.